AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_DecisionTree	Decision Tree	logloss	0.295041	5.33
	2_DecisionTree	Decision Tree	logloss	0.313541	5.19
	3_DecisionTree	Decision Tree	logloss	0.313541	5.32
	4_Linear	Linear	logloss	0.18766	6.13
	5_Default_LightGBM	LightGBM	logloss	0.125001	11.22
	6_Default_Xgboost	Xgboost	logloss	0.132326	9.43
	7_Default_CatBoost	CatBoost	logloss	0.114824	10.61
	8_Default_NeuralNetwork	Neural Network	logloss	0.187465	8.11
	9_Default_RandomForest	Random Forest	logloss	0.228603	11.64
	10_Default_ExtraTrees	Extra Trees	logloss	0.193979	10.97
	11_Default_NearestNeighbors	Nearest Neighbors	logloss	0.447072	7.98
	21_LightGBM	LightGBM	logloss	0.142061	11.57
	12_Xgboost	Xgboost	logloss	0.163252	10.06
	30_CatBoost	CatBoost	logloss	0.111399	35.02
	39_RandomForest	Random Forest	logloss	0.204734	13.2
	48_ExtraTrees	Extra Trees	logloss	0.184721	12.91
	57_NeuralNetwork	Neural Network	logloss	0.254356	9.92
	66_NearestNeighbors	Nearest Neighbors	logloss	0.488837	9.7
	22_LightGBM	LightGBM	logloss	0.119728	12.26
	13_Xgboost	Xgboost	logloss	0.132156	14.02
	31_CatBoost	CatBoost	logloss	0.116745	22.66
	40_RandomForest	Random Forest	logloss	0.213752	15.61
	49_ExtraTrees	Extra Trees	logloss	0.179832	14.59
	58_NeuralNetwork	Neural Network	logloss	0.164998	12.16
	67_NearestNeighbors	Nearest Neighbors	logloss	0.437341	11.41
	23_LightGBM	LightGBM	logloss	0.131008	15.33
	14_Xgboost	Xgboost	logloss	0.299213	12.68
	32_CatBoost	CatBoost	logloss	0.109369	19.88
	41_RandomForest	Random Forest	logloss	0.253847	16.45
	50_ExtraTrees	Extra Trees	logloss	0.254352	16.07
	59_NeuralNetwork	Neural Network	logloss	0.147187	13.83
	68_NearestNeighbors	Nearest Neighbors	logloss	0.437341	12.07
	24_LightGBM	LightGBM	logloss	0.126845	15.89
	15_Xgboost	Xgboost	logloss	0.693099	12.81
	33_CatBoost	CatBoost	logloss	0.11248	20.05
	42_RandomForest	Random Forest	logloss	0.277495	19.01
	51_ExtraTrees	Extra Trees	logloss	0.216421	16.81
	60_NeuralNetwork	Neural Network	logloss	0.248498	14.5
	69_NearestNeighbors	Nearest Neighbors	logloss	0.488837	13.75
	25_LightGBM	LightGBM	logloss	0.122106	15.44
	16_Xgboost	Xgboost	logloss	0.693123	14
	34_CatBoost	CatBoost	logloss	0.101761	19.06
	43_RandomForest	Random Forest	logloss	0.27513	18.41
	52_ExtraTrees	Extra Trees	logloss	0.240711	19.21
	61_NeuralNetwork	Neural Network	logloss	0.228268	15.97
	70_NearestNeighbors	Nearest Neighbors	logloss	0.507229	15.64
	26_LightGBM	LightGBM	logloss	0.127732	19.15
	17_Xgboost	Xgboost	logloss	0.43164	15.93
	35_CatBoost	CatBoost	logloss	0.115992	18.36
	44_RandomForest	Random Forest	logloss	0.261804	18.8
	53_ExtraTrees	Extra Trees	logloss	0.299218	22.85
	62_NeuralNetwork	Neural Network	logloss	0.176633	17.77
	71_NearestNeighbors	Nearest Neighbors	logloss	0.488837	16.6
	27_LightGBM	LightGBM	logloss	0.13886	19.74
	18_Xgboost	Xgboost	logloss	0.1746	19.75
	36_CatBoost	CatBoost	logloss	0.114289	51.53
	45_RandomForest	Random Forest	logloss	0.235061	22.4
	54_ExtraTrees	Extra Trees	logloss	0.220836	21.08
	63_NeuralNetwork	Neural Network	logloss	0.21853	18.39
	72_NearestNeighbors	Nearest Neighbors	logloss	0.489052	18.02
	28_LightGBM	LightGBM	logloss	0.116853	20.88
	19_Xgboost	Xgboost	logloss	0.691893	18.25
	37_CatBoost	CatBoost	logloss	0.11095	27.31
	46_RandomForest	Random Forest	logloss	0.195568	23.11
	55_ExtraTrees	Extra Trees	logloss	0.175504	22.43
	64_NeuralNetwork	Neural Network	logloss	0.188742	20.6
	29_LightGBM	LightGBM	logloss	0.141791	22.05
	20_Xgboost	Xgboost	logloss	0.129838	23.47
	38_CatBoost	CatBoost	logloss	0.109186	43
	47_RandomForest	Random Forest	logloss	0.276271	23.77
	34_CatBoost_GoldenFeatures	CatBoost	logloss	0.105791	33.22
	38_CatBoost_GoldenFeatures	CatBoost	logloss	0.10771	37.51
	32_CatBoost_GoldenFeatures	CatBoost	logloss	0.111014	28.44
	34_CatBoost_KMeansFeatures	CatBoost	logloss	0.100147	31.69
	38_CatBoost_KMeansFeatures	CatBoost	logloss	0.096346	51.72
	73_CatBoost	CatBoost	logloss	0.0992113	46.54
	74_CatBoost	CatBoost	logloss	0.0989191	76.72
	75_CatBoost	CatBoost	logloss	0.099829	32.25
	76_CatBoost	CatBoost	logloss	0.098331	39.46
	77_CatBoost	CatBoost	logloss	0.105146	24.75
	78_CatBoost	CatBoost	logloss	0.11485	31.32
	79_LightGBM	LightGBM	logloss	0.123575	23.95
	80_LightGBM	LightGBM	logloss	0.119156	26.91
	81_LightGBM	LightGBM	logloss	0.117737	25.96
	82_LightGBM	LightGBM	logloss	0.119728	25.24
	83_CatBoost	CatBoost	logloss	0.0934722	41.33
	84_CatBoost	CatBoost	logloss	0.103991	51.4
	85_CatBoost	CatBoost	logloss	0.10591	33.37
	86_CatBoost	CatBoost	logloss	0.0964026	57.86
	83_CatBoost_BoostOnErrors	CatBoost	logloss	0.0933352	44.61
the best	Ensemble	Ensemble	logloss	0.0914001	29.38

AutoML Performance

AutoML Performance Boxplot

Spearman Correlation of Models

models spearman correlation

	score	threshold
logloss	0.193979	nan
auc	0.974161	nan
f1	0.92779	0.512511
accuracy	0.927948	0.512511
precision	1	0.774203
recall	1	0.0166806
mcc	0.856244	0.603755

	score	threshold
logloss	0.193979	nan
auc	0.974161	nan
f1	0.92779	0.512511
accuracy	0.927948	0.512511
precision	0.929825	0.512511
recall	0.925764	0.512511
mcc	0.855903	0.512511

	Predicted as 0	Predicted as 1
Labeled as 0	213	16
Labeled as 1	17	212

	score	threshold
logloss	0.447072	nan
auc	0.971263	nan
f1	0.950749	0
accuracy	0.949782	0
precision	0.981928	0.8
recall	0.969432	0
mcc	0.900259	0

	score	threshold
logloss	0.447072	nan
auc	0.971263	nan
f1	0.950749	0
accuracy	0.949782	0
precision	0.932773	0
recall	0.969432	0
mcc	0.900259	0

	score	threshold
logloss	0.163252	nan
auc	0.981637	nan
f1	0.958057	0.40206
accuracy	0.958515	0.40206
precision	1	0.945241
recall	1	0.000494429
mcc	0.917249	0.40206

	score	threshold
logloss	0.132156	nan
auc	0.987195	nan
f1	0.961039	0.194584
accuracy	0.960699	0.194584
precision	1	0.954132
recall	1	4.70754e-07
mcc	0.921538	0.194584

	score	threshold
logloss	0.299213	nan
auc	0.960813	nan
f1	0.917749	0.406873
accuracy	0.917031	0.406873
precision	1	0.85484
recall	1	0.0303831
mcc	0.834571	0.489605

	score	threshold
logloss	0.693099	nan
auc	0.503938	nan
f1	0.666667	0.441526
accuracy	0.502183	0.490584
precision	0.511111	0.500094
recall	1	0.441526
mcc	0.00733531	0.490584

	score	threshold
logloss	0.43164	nan
auc	0.89983	nan
f1	0.836576	0.384851
accuracy	0.820961	0.51912
precision	0.929577	0.791228
recall	1	0.0919202
mcc	0.653014	0.384851

	score	threshold
logloss	0.1746	nan
auc	0.980979	nan
f1	0.946429	0.510883
accuracy	0.947598	0.510883
precision	1	0.888737
recall	1	0.00443593
mcc	0.89739	0.563837

	score	threshold
logloss	0.691893	nan
auc	0.539053	nan
f1	0.666667	0.416477
accuracy	0.524017	0.492134
precision	0.657143	0.501211
recall	1	0.416477
mcc	0.0904042	0.501211

	score	threshold
logloss	0.295041	nan
auc	0.930369	nan
f1	0.878543	0.381818
accuracy	0.877729	0.594595
precision	1	0.931818
recall	1	0.028481
mcc	0.755487	0.594595

	score	threshold
logloss	0.129838	nan
auc	0.986861	nan
f1	0.960699	0.330348
accuracy	0.960699	0.330348
precision	1	0.974077
recall	1	3.12422e-07
mcc	0.921538	0.464209

	score	threshold
logloss	0.142061	nan
auc	0.98526	nan
f1	0.965066	0.381407
accuracy	0.965066	0.381407
precision	1	0.965505
recall	1	8.07996e-12
mcc	0.930131	0.381407

	score	threshold
logloss	0.119728	nan
auc	0.98854	nan
f1	0.964758	0.574991
accuracy	0.965066	0.574991
precision	1	0.988363
recall	1	2.18901e-09
mcc	0.930273	0.574991

	score	threshold
logloss	0.131008	nan
auc	0.98792	nan
f1	0.965368	0.249998
accuracy	0.965066	0.249998
precision	1	0.943099
recall	1	4.0149e-09
mcc	0.930273	0.249998

	score	threshold
logloss	0.126845	nan
auc	0.986995	nan
f1	0.964758	0.472262
accuracy	0.965066	0.472262
precision	1	0.968365
recall	1	3.19017e-08
mcc	0.930273	0.472262

	score	threshold
logloss	0.122106	nan
auc	0.989083	nan
f1	0.969697	0.28944
accuracy	0.969432	0.28944
precision	1	0.941556
recall	1	2.72276e-13
mcc	0.939008	0.28944

	score	threshold
logloss	0.127732	nan
auc	0.987567	nan
f1	0.965066	0.389129
accuracy	0.965066	0.389129
precision	1	0.98245
recall	1	1.15357e-09
mcc	0.930131	0.389129

	score	threshold
logloss	0.13886	nan
auc	0.985832	nan
f1	0.961039	0.272588
accuracy	0.960699	0.272588
precision	1	0.974449
recall	1	1.06935e-11
mcc	0.921538	0.272588

	score	threshold
logloss	0.116853	nan
auc	0.989951	nan
f1	0.961039	0.288601
accuracy	0.960699	0.288601
precision	1	0.982453
recall	1	1.40091e-07
mcc	0.921538	0.288601

	score	threshold
logloss	0.141791	nan
auc	0.985174	nan
f1	0.969432	0.410127
accuracy	0.969432	0.410127
precision	1	0.96471
recall	1	1.65529e-11
mcc	0.938865	0.410127

	score	threshold
logloss	0.313541	nan
auc	0.95309	nan
f1	0.924078	0.266667
accuracy	0.923581	0.266667
precision	0.994652	0.818182
recall	0.9869	0
mcc	0.849166	0.6875

	score	threshold
logloss	0.111399	nan
auc	0.993745	nan
f1	0.965665	0.113672
accuracy	0.965066	0.113672
precision	1	0.954733
recall	1	3.6781e-05
mcc	0.930699	0.113672

AutoML Leaderboard

AutoML Performance

AutoML Performance Boxplot

Spearman Correlation of Models

Summary of 10_Default_ExtraTrees

Extra Trees Classifier (Extra Trees)

Validation

Optimized metric

Training time

Metric details

Metric details with threshold from accuracy metric

Confusion matrix (at threshold=0.512511)

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

Summary of 11_Default_NearestNeighbors

k-Nearest Neighbors (Nearest Neighbors)

Validation

Optimized metric

Training time

Metric details

Metric details with threshold from accuracy metric

Confusion matrix (at threshold=0.0)

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

Summary of 12_Xgboost

Extreme Gradient Boosting (Xgboost)

Validation

Optimized metric

Training time

Metric details

Metric details with threshold from accuracy metric

Confusion matrix (at threshold=0.40206)

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

Summary of 13_Xgboost

Extreme Gradient Boosting (Xgboost)

Validation

Optimized metric

Training time

Metric details

Metric details with threshold from accuracy metric

Confusion matrix (at threshold=0.194584)

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

Summary of 14_Xgboost

Extreme Gradient Boosting (Xgboost)

Validation

Optimized metric

Training time

Metric details

Metric details with threshold from accuracy metric

Confusion matrix (at threshold=0.406873)

	score	threshold
logloss	0.116745	nan
auc	0.992582	nan
f1	0.965368	0.206498
accuracy	0.965066	0.206498
precision	1	0.929169
recall	1	3.12795e-05
mcc	0.930273	0.206498

	score	threshold
logloss	0.109369	nan
auc	0.993326	nan
f1	0.965665	0.114513
accuracy	0.965066	0.114513
precision	1	0.952774
recall	1	4.25201e-05
mcc	0.930699	0.114513

	score	threshold
logloss	0.111014	nan
auc	0.991095	nan
f1	0.969432	0.222911
accuracy	0.969432	0.222911
precision	1	0.927431
recall	1	7.16241e-05
mcc	0.938865	0.222911

	score	threshold
logloss	0.11248	nan
auc	0.993421	nan
f1	0.965665	0.158107
accuracy	0.965066	0.158107
precision	1	0.92678
recall	1	3.77887e-05
mcc	0.930699	0.158107

	score	threshold
logloss	0.101761	nan
auc	0.993154	nan
f1	0.969432	0.332354
accuracy	0.969432	0.332354
precision	1	0.941953
recall	1	3.10927e-05
mcc	0.939008	0.476525

	score	threshold
logloss	0.105791	nan
auc	0.992029	nan
f1	0.969432	0.314584
accuracy	0.969432	0.314584
precision	1	0.940397
recall	1	6.77433e-05
mcc	0.938865	0.314584

	score	threshold
logloss	0.100147	nan
auc	0.994527	nan
f1	0.965368	0.193789
accuracy	0.965066	0.193789
precision	1	0.918829
recall	1	9.09971e-05
mcc	0.930273	0.193789

	score	threshold
logloss	0.115992	nan
auc	0.989512	nan
f1	0.965066	0.291756
accuracy	0.965066	0.291756
precision	1	0.984286
recall	1	3.96694e-05
mcc	0.930131	0.291756

	score	threshold
logloss	0.114289	nan
auc	0.993574	nan
f1	0.965066	0.304119
accuracy	0.965066	0.304119
precision	1	0.907739
recall	1	4.52382e-05
mcc	0.930273	0.465208

	score	threshold
logloss	0.11095	nan
auc	0.992716	nan
f1	0.965368	0.179076
accuracy	0.965066	0.179076
precision	1	0.933231
recall	1	3.11459e-05
mcc	0.930273	0.179076